| Age | Commit message (Collapse) | Author |
|---|
|
This is the first rough implementation of "b4 submit". Currently
implemented:
- b4 submit --new : to start a new branch
- b4 submit --edit-cover : to edit the cover message
- b4 submit --update-trailers : to receive latest trailer updates from
the mailing lists
- b4 submit --send : sends the messages using existing git.sendemail
configs
For details, see "b4 submit --help".
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
A lot of maintainers use patchwork alongside b4, to make it easier to
track patches and rely on some CI integration. This commit adds some
basic patchwork integration:
- on "b4 am", "b4 shazam", "b4 pr" we will mark the relevant patchwork
entries as "Under Review"
- on "b4 ty" we can set these patches as "Accepted"
- on "b4 ty -d" we can set them as "Deferred"
To make it work, the following entries must be present in the repository
used with b4:
[b4]
pw-key = (your API token)
pw-url = https://patchwork.kernel.org
pw-project = (your project, e.g. linux-usb)
pw-review-state = under-review
pw-accept-state = accepted
pw-discard-state = deferred
To get your patchwork API token, go to your patchwork profile page.
The pw-accept-state and pw-discard-state can be overridden using the
--pw-set-state flag to "b4 ty". E.g. if you wanted to mark the patches
as "Not applicable":
b4 ty -d 5 --pw-set-state not-applicable
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
Minor nitpicking by PyCharm, but not completely unreasonable.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
The mbox, am, and pr subcommands accept an mbox on stdin and extract
the message ID. When stdin.read() is called, Python assumes the
encoding is locale.getpreferredencoding(False). This may not match
the content encoding, leading to a decoding error.
Instead feed the stdin bytes to message_from_bytes(), which leads to a
decode('ASCII', errors='surrogateescape') underneath. That's
sufficient to get the message ID from the ASCII headers.
Reported-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Kyle Meyer <kyle@kyleam.com>
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
In order to avoid some of the more obscure charset encoding problems, we
switched to using as_string() for generating messages before saving them
in an mbox file. However, this uncovered a bug where the unixfrom was
not actually generated and saved, despite as_bytes() and as_string()
supposedly behaving identically.
See:
https://docs.python.org/3/library/email.message.html#email.message.EmailMessage.as_string
This commit fixes the problem by properly setting the unixfrom and using
the recommended (and hopefully less buggy) email.generator interface
when saving mailboxes.
Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
This is a kludge that I'm putting in place to avoid hitting an encoding
bug with Python. We really shouldn't be making a fake RFC2822 message
just so we can parse it again right away, so this is a temporary measure
until I refactor how it's done.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
I'm not sure if it's a good strategy to use GitHub usernames and
username@github.com addresses for this purpose, as we really need to be
able to reach someone when we process their submissions. However, for
the time being at least don't output None.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
You can now do "b4 pr https://github.com/foo/bar/pull/NNN" and have it
fetched into FETCH_HEAD. However, this also works when combined with
--explode, which is the main reason for this feature's existence.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
If the FETCH_HEAD is not signed, then keyid is going to be None. Don't
attempt to look up UIDs in such situations.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
I've been working on a way to automatically convert pull requests into
series, complete with mailing them out to arbitrary destinations. This
would allow folks to send a pull request to a dedicated list and have it
automatically converted into a well-formed series.
This is a tentative implementation that relies on git-send-email to do
most of the heavy lifting. I have misgivings about using git-send-email
for this purpose, but it does reduce the amount of duplicated code we
would have otherwise had to write, and allows us to hook into things
like tocmd/cccmd, etc.
For example, adding the following to your .git/config:
[sendemail "autopr"]
smtpserver = [your.server.here]
smtpserverport = 587
smtpencryption = tls
smtpuser = [your-user]
smtppass = [your-pass]
transferEncoding = 8bit
suppressFrom = yes
confirm = never
validate = no
tocmd = "$(git rev-parse --show-toplevel)/scripts/get_maintainer.pl --norolestats --nol"
cccmd = "$(git rev-parse --show-toplevel)/scripts/get_maintainer.pl --norolestats --nom"
This would allow doing the following:
b4 pr -e -f "AutoPR Exploder <autopr@yourdomain.here>" -s autopr [--dry-run]
The pull request will be exploded into a patch series and sent to all
the proper destinations as returned by get_maintainer.pl. We construct
the message headers in a way that allow regular code review and "b4 am"
usage after the auto-exploded series is sent out.
If testing goes well, we'll implement this as a kernel.org service and
then hook a similar implementation via Gitlab/Github.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
This moves maildir saving code into __init__.py so that we can benefit
from it via other subcommands, such as pr.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
Normally, -f would be 'Some Service <service@example.org>', but in case
it's just "service@example.org", wrap it in angle brackets properly.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
When we're retrieving linked messages, make sure we don't clash with the
variable holding the overall mbox.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
While trying to figure out some odd DKIM failures, I've discovered that
there is an important incompatibility between git's idea of what "mbox"
format is, and Python's mboxo implementation -- at least when it comes
to treating "\nFrom " escapes.
According to the "original mbox" standard, when a message body contains
a "\nFrom " sequence, it should be converted to "\n>From " in order not
to confuse the parser. When reading messages in that format, clients are
supposed to back-convert "\n>From " into their original form. This is
the so-called "mboxo" format, which is what Python's mailbox.mbox
supports:
https://docs.python.org/3/library/mailbox.html#mailbox.mbox
The "mboxrd" format was created to avoid a corruption problem whereas a
body that legitimately contains "\n>From " would be wrongly converted
into "\nFrom " upon parsing the mailbox, so mboxrd standard requires
that, when saving a mailbox, "\n>From " sequences are additionally
escaped as "\n>>From ". This is the format public-inbox supports, so
when we grab mailboxes from remote, they are in mboxrd format.
Git will try to guess the format of the mbox file, but it will ONLY
back-convert "\n>From " sequences when you specifically tell it that
it's "mboxrd" format, even when it's in fact "mboxo":
git am --patch-format=mboxrd
If you don't force the mboxrd format, git-am will preserve all escaped
"\n>From " lines as-is.
We've been previously operating on the assumption that git-am's mbox
support properly implements "mboxo", but this was wrong, resulting in
some commits like the following:
https://git.kernel.org/torvalds/c/137733d08f4a
This large-ish change ditches all internal use of Python's mboxo. When
asked to save mbox files, we will save them without any escaping, the
way git-am (i.e. git-mailsplit) espects them. The same goes when we're
outputting to stdout.
There is also a way now to pass -M to both "b4 am" and "b4 mbox" that
will save things as maildirs -- git-am supports this natively and thus
avoids any possible parsing ambiguities. You can set a config option
b4.save-maildirs=yes to make this the default behaviour.
The fallout of this is fairly benign, if annoying. There is no situation
in which a patch would have "\nFrom " as part of its body, so the
problem only affected commit messages. We will have a handful of these
sprinkled around the trees, and will hopefully not introduce any new
ones once everyone switches to the b4 version that outputs things in the
format git-am expects.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
Move end-to-end attestation code into its own library: patatt. See
https://git.kernel.org/pub/scm/utils/patatt/patatt.git/about/
It is included into b4 as a submodule, but you will need to init it
first:
git submodule update --init
This change significantly simplifies our attestation code, dropping
thousands of lines of rather hairy code. Notably, patatt-style
attestation is incompatible with previous attestation implementations
done directly in b4, but that's just as well -- we've always marked it
as "experimental" and the lack of adoption was proving that we weren't
on the right path.
Next to come is keyring management and documentation.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
If we clean the to/cc headers to get rid of all unicode escaping, we run
into a Python bug that is unable to properly parse addresses, e.g.:
In [5]: from email import utils
In [6]: utils.getaddresses(['foo <foo@bar.com>'])
Out[6]: [('foo', 'foo@bar.com')]
In [7]: utils.getaddresses(['Shuming [范書銘] <shumingf@realtek.com>'])
Out[7]:
[('', 'Shuming'),
('', ''),
('', '范書銘'),
('', ''),
('', 'shumingf@realtek.com')]
If we store the headers as-is from the original message, we are less
likely to run into this bug, as all non-ascii sequences should be
qp-escaped in the original headers:
=?big5?B?U2h1bWluZyBbrVOu0bvKXQ==?= <shumingf@realtek.com>
This doesn't fix the underlying bug in Python, but works around it.
Reported-by: Mark Brown <broonie@kernel.org>
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
When we come across a pull request that doesn't list its base (i.e. not
generated using git-request-pull), we try to figure out the base
ourselves by doing merge-base. However, if the pull request is already
merged, then that is going to be a useless operation, so recognize this
situation and bail out early.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
It's a bad habit to exit anywhere other than from main() anyway. I
should fix all cases of that in order to be both more pythonic and
library-friendly.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
We probably want to be able to tweak the output of git-format-patch
based on which list we're running it for (e.g. passing --minimal or
--histogram), so make it possible to pass extra parameters to the git
command.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
If we're not passing -g to "b4 pr -e", then we should try to see if we
are inside a git checkout and use that as our source.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
Two services we'll be running in the near future:
1. Transparency log for all pull requests
2. Auto-exploder for pull requests that can send auto-exploded patches
to all the same recipients.
This requires quite a bit more testing and refinement, but the core of
the functionality is there.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
We moved pgp sig verification code around, so fix it for the invocation
in b4 pr.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
They are much more useful if b4.thanks-commit-url-mask is set, but even
without them this will list all subjects and corresponding commit IDs.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
New experimental feature that aims to simplify a very common "thanks,
applied" kind of feedback often expected of maintainers. Still needs
documentation to explain its usage.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
When a pull request is generated using git-request-pull, the base commit
is specified in the body of the message. However, in other cases this
information can be missing, so we can use git merge-base to find out
what it is after we've performed a "git fetch".
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
We need a valid mboxo format for python, so make sure to escape all
From lines not actually starting a new message, per mboxo requirements.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
Since we aren't planning on sending any of this as actual emails,
convert EVERYTHING into utf-8 before writing messages, including
QP-escaped 7-bit headers.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
We now use similar subroutines for checking signatures on FETCH_HEAD as
we do for patch attestation, making it a convenient operation during the
fetch stage:
$ b4 pr 202003292114.2252CAEF7@keescook
Looking up https://lore.kernel.org/r/202003292114.2252CAEF7@keescook
Grabbing thread from lore.kernel.org
Looking at: [GIT PULL] seccomp updates for v5.7-rc1
Fetching https://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git tags/seccomp-v5.7-rc1
---
[✓] Attestation-by: Kees Cook <keescook@chromium.org> (pgp: 8972F4DFDC6DC026)
---
Successfully fetched into FETCH_HEAD
Hopefully, I didn't introduce too many bugs into patch attestation,
since I had to rewrite the backend a bit to work for both native git
operations and patch attestation calls.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
Fix some leftover bugs from prototyping stage when we were always
assuming to run from a checkout.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
No need to add an extra From: into the body of the message if it's
identical to the From in the original pull request.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
Fix a few most glaring problems performing explode on a message that was
not formatted using git-request-pull.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|
|
While working on "pull-request exploder" stuff for achiving on
lore.kernel.org, I realized that this may be a useful set of features
for developers as well, so here is a very simple framework for handing
pull requests. Examples:
b4 pr <msgid>
- downloads that message
- parses the pull request
- checks that the remote tip is where the message says it should be
- makes sure the base-commit is present in the tree
- makes sure the tip commit isn't already in one of the branches
- checks if FETCH_HEAD already is at that commit
- if the above two checks pass, performs a git fetch
b4 pr --check <msgid>
Runs all of the checks above, but doesn't perform the actual fetch.
Useful if you don't remember if you've already processed a pull
request or not.
b4 pr --explode <msgid>
Runs basic sanity checks and then:
- performs a git fetch
- runs a "git format-patch" for base-commit..FETCH_HEAD
- adds the same to/from/cc headers as in the pull request
- properly threads each patch below the pull request
- saves that into a <msgid>.mbox file
The above is handy if you want to comment on something in a pull
request and not have to hunt around for sender/cc information.
Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
|