-
Notifications
You must be signed in to change notification settings - Fork 95
Auto inline images #302
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Auto inline images #302
Conversation
Some general remarks:
As someone who does not use images, what problem does this solve? |
image.c
Outdated
@@ -16,7 +16,7 @@ static int image_index = 0; | |||
|
|||
/* display image */ | |||
|
|||
typedef struct _termialImage { | |||
typedef struct _termialImage /* [sic]? */ { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just fix it.
main.c
Outdated
@@ -127,7 +127,7 @@ static int searchKeyNum(void); | |||
#define help() fusage(stdout, 0) | |||
#define usage() fusage(stderr, 1) | |||
|
|||
int enable_inline_image; | |||
int enable_inline_image = -1; /* TODO: can't we remove this ifndef USE_IMAGE? */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't see an USE_IMAGE here and probably no ;)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's unused without USE_IMAGE, but you also have to ifdef out a few enable_inline_image checks (and the CLI param assignments).
inline_img_protocol_autodetect.c
Outdated
@@ -0,0 +1,127 @@ | |||
/* The only externally visible symbol exported by this module is |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This code should follow the same style as the code in other units. And I think a better place for this is image.c
inline_img_protocol_autodetect.c
Outdated
@@ -0,0 +1,127 @@ | |||
/* The only externally visible symbol exported by this module is | |||
* inline_img_protocol_autodetect(). I appreciate that this name does not | |||
* roll off the tongue |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are you trolling?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No? I certainly intended no offence, I must have just been in a good mood while writing that, sorry
main.c
Outdated
@@ -720,6 +720,8 @@ main(int argc, char **argv) | |||
} | |||
} | |||
#endif | |||
else if (!strcmp("-auto-inline-img", argv[i])) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this need to have a command line switch? It's should be an option anyways and the can be set via -o.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I considered that, but thought that inline_img_protocol=5
or enable_inline_image == 5
would imply a built-in protocol rather than use of w3mimgdisplay for any code looking for a positive value to mean that, while auto-detection can resolve to 0 (ie. use w3mimgdisplay). Also, if a new image protocol were to become relevant, such as Terminology's, having auto-detection in the middle of a list of protocols would make no sense. Trouble is, set_param() doesn't allow negative values for P_INT and I thought changing that could have excessively far-reaching implications for anything assuming P_INT
was >= 0.
Looking at set_param again, this could fixed by making auto_inline_image with P_SHORT or P_CHARINT and changing the type of enable_inline_image
accordingly, but at the time I just went for the easiest and simplest way so I could test it
It avoids having to manually set the inline image output method (since different methods work best depending on your environment). Normally you'd have to go through each option yourself; this patch does the detection automatically. Some comments on the implementation:
|
Ah my apologies, that's my bad habit of committing WIP as I go so I remember what order I've done things in. But can't the merge simply be squashed? Or should I rebase and squash it myself and open a different PR? (with changes moved to image.c ofc)
As far as I could see,
Not every user is aware of the different protocols available, nor which if any are supported by their terminal -- I was unaware that more recent Konsole versions support sixel, iterm2 and kitty images, since I'd been using it since before those versions, and xterm and xfce4-terminal both have undocumented sixel support. Of course, on Wayland the w3mimgdisplay X11 fallback doesn't work Also, for |
I don't think so, get_pixel_per_cell uses XTWINOPS
I'm not sure existing config would be broken -- if inline_img_protocol is set in a config, it would override default autodetection. I would have liked to use |
Ah, no, I meant sending them together in a single batch. Terminal query code usually does this in one pass, putting primary DA in the end, because one write/read/timeout cycle will always be faster than two. (Also, pretty much every terminal emulator responds to a primary DA but not necessarily to other queries, so sending it as the final query is a nice timeout prevention measure.) We can live without that for now, since we only have two queries, both seem to be widely supported, and get_pixel_per_cell often returns without sending a query at all. Just a suggestion for improvement.
Yeah, I now see you're right. The only problem with the current solution remains that it doesn't let the user pick "autodetect" and then stick with it, which would be nice for when you use w3m in different terminals. (Even if you could pick -1 in its current form, it would automatically switch to another option as soon as you change something else in the option setting panel.) Hence my suggestion to use a separate internal variable for the actual image protocol choice. |
* Remove iterm2 test * Move image protocol detection functions into image.c * Replace -auto-inline-img with -o inline_img_protocol=-1, by changing inline_img_protcol to a P_CHARINT * Test for presence of img2sixel command before allowing sixel protocol selection
In the absence of any further comments on the clutter of the commit history, I'll just push any further commits and potentially clean up history later (I know `clean up later' is never a good thing to hear ;). The diff against master still looks fairly clean. This implements most of the changes just discussed
Oh I get it
I'm afraid I don't follow here -- by the time you get to the option setting panel, if autodetect was set it will already have run at startup and resolved to a different value INLINE_IMG_*. Or do you mean it should not run only at startup? The way I see it right now, if the user picks a different setting for inline_img_protocol, this by design should override the auto-detected value |
The auto detect should be an option in the options panel, so that a user does not have to use the command line option in order to enable it. Re: commit history, you can squash the commits and just do a force push into your branch. Remember to also update the docs, when you are done with the implementation. |
Yeah, that's what I meant. You could do something like:
So if
What docs? :P OK, there's doc/README.sixel, which is a bit outdated and only talks about the sixel method. Also, doc/README.img, which only talks about the X/framebuffer method. I think it would be best to merge the two, and also add some info on the other methods (+ how to select them). It's kind of my fault for never updating the docs when working on this, so I'll prepare a PR for it once this patch is finalized. |
On Mon, Jul 22, 2024 at 01:32:28PM -0700, bptato wrote:
> Remember to also update the docs, when you are done with the implementation.
What docs? :P
Well, adding a command line option should at least update the man page
and probably also the MANUAL.html.
|
Wait what? I thought we decided we don't need a command line option... Looking at the man page and MANUAL.html, neither did ever mention image support. I've started working on an overhauled README.img, I'd say it's easiest to just link to that from MANUAL.html once it's finished. |
On Tue, Jul 23, 2024 at 07:43:47AM -0700, bptato wrote:
> Well, adding a command line option should at least update the man page
> and probably also the MANUAL.html.
Wait what? I thought we decided we don't need a command line option...
Did we? That's good, because that's what I preferred. :-D
Looking at the man page and MANUAL.html, neither did ever mention
image support. I've started working on an overhauled README.img, I'd
say it's easiest to just link to that from MANUAL.html once it's
finished.
Sounds good.
|
3629d57
to
7d07f7f
Compare
Alright, I've squashed the commits from up til now, and the second commit adds the separate variables for Only issue is now the user can't see in the options panel which protocol was selected automagically, but that's not the end of the world. If it does become necessary, maybe that information would be more suited to debug output on stderr anyway. Also, looks like |
JFTR, I have this on my radar. In the current form it's not easy to review, because there seem to be a lot of unrelated changes mixed into the commits, but I will try to work it out when my time permits. |
Is this still valid? If yes, can you make that improvement @el-remph , please. |
Sorry I was away, here's a mockup that seems to work, but I'm not sure if this does optimise anything after all, because to know if we need pixel/char we need to know whether we're using enable_inline_img or not, so (if kitty and mlterm protocols are unavailable) we need to know if the terminal supports sixel before we query pixel/char. (There's more that could be done to make this prettier, but I'm leaving that until I know if this optimises anything or not) My 'solution' in this commit, is just call get_pixel_per_char() unconditionally, which will lead to an unnecessary check for pixel/char for those without any supported inline image protocol. On most systems, I don't think this will use XTWINOPS, since TIOCGWINSZ is usually available, so this works out to an extra ioctl call for the typical (I guess) user. I don't know the cost of ioctl so this may not be much, but the only benefits are for those on systems without TIOCGWINSZ so I doubt it's worth it. Unless anyone has any ideas, I'll rollback this commit |
When I wrote "We can live without that for now", I meant it. It isn't that relevant with just two queries, especially when one of those is just a fallback. The current (as in, before this patch) pixel per column/line logic seems flawed anyway, as the same variables are used for tables too ( |
239e481
to
7d07f7f
Compare
7d07f7f
to
31c042f
Compare
Rebased with detection of ghostty, which has had support for kitty images since very early on afaict. Anyone know the status of this pr? |
image.c
Outdated
/* the string in the following sizeof expression is what I think the | ||
* largest possible response could be. See | ||
* https://invisible-island.net/xterm/ctlseqs/ctlseqs.html | ||
* for more information, specifically the section on CSI Ps c ("\e[c") */ | ||
char term_response[sizeof "\033[?65;1;2;3;4;6;8;9;15;16;17;18;21;22;28;29c"] = ""; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FYI, this is by no means the largest possible response. The list of features on the xterm site are just the ones that xterm supports. DEC documented more than 40 of them, and some modern terminals have defined a few of their own.
That said, no one terminal is likely to report all of them, but there will definitely be terminals with longer responses than the one above. For example, this is an example from a real DEC VT520:
\033[?65;1;2;6;7;8;9;12;15;18;19;21;23;24;42;44;45;46c
A VT525 has at least one more feature than that, so would be even longer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. Plus read
isn't guaranteed to read the entire response in the first place.
It's better to loop until you see a c.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should also address the problem of snarfing up input on museum pieces, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No. Terminal querying is inherently racy, and the only comprehensive solution to that is to feed back any unexpected data before & after the response to an internal buffer (which w3m does not have) for subsequent processing.
In practice, I doubt anybody will run into this - try using images over a high-latency connection to see why :P
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well then, still using a loop and a dynamically sized buffer seems the right approach here.
On Wed, Jan 15, 2025 at 07:10:09PM -0800, el-remph wrote:
[...]
Anyone know the status of this pr?
I didn't follow the last changes that closely, but I'd be willing to
accept this into my fork when all current issues are addressed. I'd like
to have an OK from bptato and maybe from j4james first though.
Also I'd prefer if you would send it the mailing list[1] then I will do
the final review.
Chances that it will be merged here are next to zero, I guess.
[1]: ~rkta/[email protected]
|
image.c
Outdated
fd_set fds; | ||
struct timeval timeout = { 0, 0.2 * 1000000 }; /* wait 0.2s, from terms.c */ | ||
FD_ZERO(&fds); | ||
FD_SET(STDIN_FILENO, &fds); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just realized you're reading from stdin - that won't work: echo test | w3m
Please move the function to terms.c and use the tty
variable, like get_pixel_per_cell does.
image.c
Outdated
_exit(-1); | ||
default: | ||
return | ||
#ifdef HAVE_WAITPID |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is 2025, I feel we can afford to assume waitpid exists :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To be fair, while you are right, this just reuses the pattern used in the rest of the code base. I will need to address this anyways.
Maybe it's OK to keep it for consistency, I don't really care.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I still think, this needs to be documented somewhere.
image.c
Outdated
static int | ||
protocol_test_environment(void) | ||
{ | ||
const char * envptr; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
*envptr
image.c
Outdated
const char * envptr; | ||
|
||
if ((envptr = getenv("TERM"))) { | ||
/* The purpose of strncmp(3) where used below is like the glob `prefix*', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Commentstyle for multi-line comments should be
/*
* comment
*/
image.c
Outdated
_exit(-1); | ||
default: | ||
return | ||
#ifdef HAVE_WAITPID |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To be fair, while you are right, this just reuses the pattern used in the rest of the code base. I will need to address this anyways.
Maybe it's OK to keep it for consistency, I don't really care.
image.c
Outdated
/* the string in the following sizeof expression is what I think the | ||
* largest possible response could be. See | ||
* https://invisible-island.net/xterm/ctlseqs/ctlseqs.html | ||
* for more information, specifically the section on CSI Ps c ("\e[c") */ | ||
char term_response[sizeof "\033[?65;1;2;3;4;6;8;9;15;16;17;18;21;22;28;29c"] = ""; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should also address the problem of snarfing up input on museum pieces, right?
image.c
Outdated
* https://invisible-island.net/xterm/ctlseqs/ctlseqs.html | ||
* for more information, specifically the section on CSI Ps c ("\e[c") */ | ||
char term_response[sizeof "\033[?65;1;2;3;4;6;8;9;15;16;17;18;21;22;28;29c"] = ""; | ||
char * ptr; /* general-purpose pointer to the above buffer */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
*ptr, the asterisks sticks to the name in C.
image.c
Outdated
FD_ZERO(&fds); | ||
FD_SET(STDIN_FILENO, &fds); | ||
if ((nret = select(STDIN_FILENO + 1, &fds, NULL, NULL, &timeout)) <= 0) { | ||
fprintf(stderr, "%s: %s\n", errstr, nret == 0 ? "timed out" : strerror(errno)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please keep it below 80 chars, if there is no very good reason to do otherwise.
Applies to other spots as well.
image.c
Outdated
|| memcmp(term_response, "\033[?", 3) != 0 | ||
|| !(ptr = memchr(term_response, 'c', nchars_read))) | ||
{ | ||
fputs("Malformed terminal attributes\n", stderr); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is writing to stderr good practice here or should this use disp_err_message?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't think it was worth displaying it prominently to a user, just outputting the information to anyone who is looking for it. Writing to stderr seems fairly common throughout the codebase
image.c
Outdated
if (*++ptr != '4') | ||
continue; | ||
switch (*++ptr) { | ||
case ';': case 'c': |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't hide the case 'c':
on the same line. Put it on a separate line, please.
@@ -449,7 +450,7 @@ struct param_ptr params1[] = { | |||
CMT_EXT_IMAGE_VIEWER, NULL}, | |||
{"image_scale", P_SCALE, PI_TEXT, (void *)&image_scale, CMT_IMAGE_SCALE, | |||
NULL}, | |||
{"inline_img_protocol", P_INT, PI_SEL_C, (void *)&enable_inline_image, | |||
{"inline_img_protocol", P_CHARINT, PI_SEL_C, (void *)&enable_inline_image_config, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is P_CHARINT needed here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because P_INT does not accept negative values.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, it's ignored when setting the value. I only looked at reading it... Bad naming though...
But then OK.
image.c
Outdated
if (strncmp(envptr, "mlterm", 6) == 0) | ||
return INLINE_IMG_OSC5379; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not in a position to test, but I was under the impression that MLterm had dropped support for OSC 5379 images several years ago. See the 3.8.5 release notes.
I may be misinterpreting what they're saying there, but I think it would be good idea to confirm this actually works (if you haven't already done so) before forcing the use of OSC 5379. MLterm does support Sixel, so you should still have that as a fallback.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Huh, indeed. It's not fully gone, but you have to pass a flag when compiling:
./configure CFLAGS=-DENABLE_OSC5379PICTURE
I suppose it's better to stick with Sixel.
31c042f
to
f2a180d
Compare
Rebased with the changes recommended:
Oh, I didn't realise about the mailing list. If these changes pass muster here then I'll submit it to the mailing list |
I've tested the latest version:
Well, but the image docs in your fork are now different... I can update them in a separate patch, if you're OK with that. |
@bptato I don't think accepting the |
Ah, no, by "support" I meant "don't print an error and stop looping" :P SyncTERM (at least the version I have, from a few months ago) doesn't do private color registers, so you have to manually configure W3M_IMG2SIXEL for it anyway. |
f2a180d
to
53124af
Compare
@bptato Yep that's obscure alright (:
afaict activeImage is only set to TRUE once and never set back to FALSE, so if w3m is started and initImage called without auto-detection, and the user then enables auto-detection, that won't work. I haven't been able to solve this. One option could be to cache the result of auto-detection once it's done, then reuse that on subsequent calls -- is it possible for the result of auto-detection to change during a session? For example, if the user starts w3m in GNU screen in a non-sixel terminal, then reattaches in a sixel terminal? Is it possible for the TERM environment variable to be modified during a session? |
Ah, I see. I would have expected that I have to restart w3m, but it's definitely nicer if it works immediately :)
On GNU screen, img2sixel only seems to work with the "penetrate" hack. (But maybe I'm missing something.) For a working example, see tmux: it reports Sixel capability even on terminals that cannot display Sixel, so you can attach both a Sixel-capable and a non-Sixel-capable terminal at the same time (and see a placeholder when appropriate). In other words: I wouldn't worry about this.
No. |
…inal * Compile out enable_inline_img ifndef USE_IMAGE * Add enable_inline_img_config for configuration; leave enable_inline_img for state * Make auto-selection of inline_img_protocol the default value of enable_inline_img_config
53124af
to
ce410b1
Compare
Deduce an appropriate value for inline_img_protocol rather than requiring it to be set manually with eg.
-o auto_inline_img=4
for kitty.Tangentially, should
struct _termialImage {...} TerminalImage
at image.c:11,27 be_terminalImage
? Not important currently since nothing uses the struct tag as opposed to theTerminalImage
typedef, but could surprise someone eventually