Subject: Scripting Support for Screen

From: Sadrul Habib Chowdhury <[email-masked]>
Date: Thu, 24 Jul 2008 12:00:04 -0400

Hi. I started working on a lua script loader for screen last night. I am
quite happy with the progress so far. I would like to know what people
think about this kind of work. I discussed about this briefly with Micah
last night, and he also thought discussing this in the list would be
useful.

I will try to answer a few questions that might come up in advance:

* Where is the patchset?
- The current patchset is available at
  http://www.pidgin.im/~sadrul/pp/scripting-lua/. The patches are against
  the development version of screen
  (http://git.savannah.gnu.org/gitweb/?p=screen.git)

* What's the goal?
- The goal right now is to allow executing some user-defined function
  on specific events, e.g. when a window is closed, opened,
  title/position changed, a display is detached etc. etc. easily, i.e.
  without requiring a recompilation/reinstallation.

  I started working on this mostly out of curiosity, to see where it
  goes, and what cool things it might allow users to do. One use I have
  in mind is to send some signal to the process in a window when it's not
  visible in any of the displays, (and send another one when it becomes
  visible for the first time). It doesn't make all that much sense to
  implement such highly personalized features in screen itself.

  I am sure people will find a lot of other interesting things to do.

* Why Lua?
- Because I like it!
  To be perfectly honest, there was no compelling technical reason for me
  to choose lua. I hadn't worked with lua bindings before, but I was told
  it's much easier to work with than with some other scripting languages
  (e.g. python!). I thought I would give it a try and find out.

* Why not guile?
- Because I haven't seen it before, and I wasn't aware of it until Micah
  told me about it last night on IRC!

  Note: with the patchset, it is possible to support multiple scripting
  languages, either built-in at compile time (current), or dynamically
  loaded on demand (not yet implemented, but possible semi-trivially.
  [1] for details). So in theory, it will be possible to have none,
  either or both lua and guile support. (and any other language someone
  cares to write bindings for)

* What about the bloat?
- I would refer to the commit message at [1]

* Is there a sample script?
- Yes! http://www.pidgin.im/~sadrul/pp/scripting-lua/sc.lua
  The 'fore_changed' function towards the end is the one that gets
  executed when the foreground window for a display is changed. It prints
  the list of windows (# and title), a message explaining the switch from
  the previous window to the current one, and finally prints detailed
  information about each display and its foreground window. The display
  information includes the term, tty, user and the canvases.

* What are the current triggers/hooks?
- Right now, only changing the foreground window in a display will
  trigger a function named 'fore_changed' in the lua script. I don't have
  a complete list of hooks in mind yet. I plan on adding new hooks as the
  need arises. Suggestions are very welcome!

I would love to know if people have any opinions, suggestions etc. about
this.

[1] http://www.pidgin.im/~sadrul/pp/scripting-lua/0003-Add-a-layer-of-abstraction-for-the-script-bindings.patch

Cheers,
Sadrul

_______________________________________________
screen-users mailing list
[email-masked]
http://lists.gnu.org/mailman/listinfo/screen-users

---

From: Micah Cowan <[email-masked]>
Date: Thu, 24 Jul 2008 12:07:38 -0700

Johannes Weiner wrote:
> The trend I would like to see is not just extending screen with a 
> powerful language but reducing the C code where possible by this 
> language.  Sort of like it's done with emacs (only that emacs was 
> probably written to this design from the beginning, I assume).
> 
> Because the more you do in extension language, the more powerful 
> commands you can provide to the scripting interface.
> 
> And the more your configuration file becomes a program on its own, 
> written for the screen library :)

I agree with all of this.

I mentioned to Sadrul that I find Guile preferable (which would make
Screen decidedly Emacs-like); mainly because everything is an expression
(everything's a list), so you can use for-loops inside the dynamic value
for hardstatus, etc. As I've looked at our current String Escapes
support and the sorts of features people have been requesting in String
Escapes, it's becoming clear that the current state, and any additions
we make atop it, are an ugly hack that really wants to be full
expressive support. Rather than using %w and %W and adding flags to
change their behavior, and then being dissatisfied when we can't do what
we want (restrict to specific window flags, doing "odd/even" coloring of
window titles, add "blink" rendition to windows with the alert flag), we
should be looping over the windows and examining the attributes we want,
to generate the appropriate string.

Supplying "hooks" to screen as a C API would make it possible to tack on
any scripting lanuguage we choose, which is worth considering. However,
I'd like to see direct support for some scripting language within
screen; Guile looks like a good choice to me.

Your comments regarding "reducing C" are appropriate; though I'm not
sure I agree with it literally, as it might impede Screen's efficiency,
and adding as little crawl beyond the "real" terminal's processing time
is a worthwhile value. But having pretty much everything be
accomplishable via the scripting language seems desirable.

> Sort of like it's done with emacs (only that emacs was probably 
> written to this design from the beginning, I assume).

GNU Emacs was designed this way from the beginning. But this was based
on the community's past experience with Emacs; the original Emacs
was not built around a Lisp engine. Instead, it was based on a very
clunky language, that was sort of incrementally hacked to support the
various whims of its users. It was a similar situation to our String
Escapes, in that the original language lacked looping constructs (it
looks like it may have been roughly similar to the "ed" set of line
editing commands) until loops were hacked in (as the < and > commands).
It was the experience of the user community trying to hack
programming-like applications in a non-programmer-friendly language that
led to the conclusion that the best way to go was to make an
Emacs with a full-fledged programming language at its heart. Bernie
Greenberg's Emacs for Multics was the first Lisp-based Emacs (it was
written entirely in Lisp).

(http://www.gnu.org/gnu/rms-lisp.html was the primary reference I used
for this information.)

- --
Micah J. Cowan
Programmer, musician, typesetting enthusiast, gamer,
and GNU Wget Project Maintainer.
http://micah.cowan.name/

---

From: "Tom Scogland" <[email-masked]>
Date: Thu, 24 Jul 2008 15:47:39 -0500

On Thu, Jul 24, 2008 at 2:07 PM, Micah Cowan <[email-masked]> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Johannes Weiner wrote:
While I'm relatively new here I thought I'd throw in my 2 cents on the
matter. Having everything doable via scripting sounds good, but it should be
a situation where something is only done in scripting if you want to replace
the default so as to reduce the overhead for the people that just want to
use the defaults. Or as Micah pointed out, things that currently are handled
poorly, and would significantly benefit from scripting without adding too
much overhead (string escapes etc.).

As far as what language to use, I know lisp and guile and the like are
classic choices, but might it be a good idea to make it a language people
are likely to know outside of the context of scripting? I'm not saying it
has to be, but something in the python/perl/ruby/haskell family might be a
good idea if only because people know them well already.  It's like the
experience with vim, it has a primary scripting language, which is different
from what most people know, which people used because it's what was there,
but finally python support went in and several complex extensions came into
existance which could not have otherwise.  Just something to consider.
(NOTE: I'm not a python user personally, just used as an example)

> (http://www.gnu.org/gnu/rms-lisp.html was the primary reference I used

All of this is pretty correct, except that the gnu emacs we know was lisp
with a C core for a long time because redraw was impossible to write
efficiently enough in lisp for some reason.  As such, when the program
entered that state, everything changed, and that's something that should
probably be avoided.  Either have it be all scripted, or have it be a C core
which can be extended and modified from outside, but avoiding a situation
where most of the program is one way and certain parts are another, that
will just generate frustration and maintenance problems.

A final word here, bloat=bad.

I would really like to see scripting, but if it means an emacs-like
distribution of 100+ MB of scripting files and the generation of a program
which does everything well except what it was designed for, then the point
has been missed.

> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.7 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFIiNL57M8hyUobTrERArJfAJ9Bc912lIxVfB1HGE8jtcDsCiI8jgCfZjyn
> dIALgW9PktXYelZlfluLVgo=
> =+PBO
> -----END PGP SIGNATURE-----
>
>
>

-- 
-N
AKA:Tom Scogland
I am enough of an artist to draw freely upon my imagination. Imagination is
more important than knowledge. Knowledge is limited. Imagination encircles
the world.
-Albert Einstein
_______________________________________________
screen-users mailing list
[email-masked]
http://lists.gnu.org/mailman/listinfo/screen-users

---

From: Micah Cowan <[email-masked]>
Date: Thu, 24 Jul 2008 14:16:30 -0700

Tom Scogland wrote:
>> matter. Having everything doable via scripting sounds good, but it
>> should be a situation where something is only done in scripting if you
>> want to replace the default so as to reduce the overhead for the people
>> that just want to use the defaults. Or as Micah pointed out, things that
>> currently are handled poorly, and would significantly benefit from
>> scripting without adding too much overhead (string escapes etc.).

Right. That's essentially what Emacs does: use C implementations where
efficiency is important, but allow users to override with Lisp

>> classic choices, but might it be a good idea to make it a language
>> people are likely to know outside of the context of scripting? I'm not
>> saying it has to be, but something in the python/perl/ruby/haskell
>> family might be a good idea if only because people know them well
>> already.

I'm not sure I agree on "people know them well" for Haskell. Scheme
/Lisp probably has a larger developer base than Haskell does.

The only reasons I prefer Guile over those, is (1) it's much more
straightforward to use complex constructs, such as loops, within an
expression, and (2) GNU recommends it as first choice for scripting in
GNU projects (it's a GNU project, so would be "eating our own dogfood").

What I mean about complex constructs in expressions, is that in many
other languages, iterative constructs such as for-loops are statements,
not expressions. So, for instance, setting "hardstatus" to an expression
that dynamically generates a string by iterating over the window-list is
straightforward. In some of other languages, it might be impossible.
Perl could do it, probably with grep and map; Python could probably do
something similar. It wouldn't be a problem in Haskell. I don't know
about Ruby. But I suspect only in Guile and Haskell would it look like
something other than a bit of a hack.

>> It's like the experience with vim, it has a primary scripting
>> language, which is different from what most people know, which people
>> used because it's what was there, but finally python support went in and
>> several complex extensions came into existance which could not have
>> otherwise.  Just something to consider. (NOTE: I'm not a python user
>> personally, just used as an example)

AFAICT, that's more an argument for using a full-featured language, than
any particular language.

Part of the philosophy that went into creating Guile was that by
providing a powerful language, people could implement their favorite
languages in Scheme, and so people could work in whichever language they
like best. That sounds nice in theory, but as far as I know no one has
actually implemented their favorite languages in Guile, probably because
it wasn't worth the effort, so that's not much of a boon. :)

>> distribution of 100+ MB of scripting files and the generation of a
>> program which does everything well except what it was designed for, then
>> the point has been missed.

I disagree that Emacs doesn't excel at what it was designed for; but it
obviously excels at many things that are entirely unrelated to editing,
as well.

I suspect we don't have to worry too much about that for Screen; but
part of that may depend on how choosy we are about what we let into the
Screen distribution. For my part, I don't currently see any reason why
we would need to provide any Scheme code with Screen whatsoever, apart
from probably a sample ~/.screenrc.scm, and perhaps other example
scripts. Sure, a powerful programming language means that folks could
write "Towers of Hanoi" or an email client within Screen; but that
doesn't mean we have to include it. And anyway, why would they want to
do that when they could just do it in Emacs?

To be honest, implementing a Screen within Emacs makes almost as much
sense as giving Screen Emacs-like scriptability; Screen has already
duplicated quite a bit of Emacs' layout functionality and such, some of
it not yet as well as Emacs itself does it. But I doubt anyone's
interested in seeing that happen. :)

- --
Micah J. Cowan

---

From: Micah Cowan <[email-masked]>
Date: Thu, 24 Jul 2008 14:25:01 -0700

Micah Cowan wrote:
> sense as giving Screen Emacs-like scriptability

Come to think of it, that functionality more-or-less already exists;
"M-x term" within Emacs pretty much fits the bill.

- --
Micah J. Cowan

---

From: "Tom Scogland" <[email-masked]>
Date: Thu, 24 Jul 2008 16:32:40 -0500

> /Lisp probably has a larger developer base than Haskell does.

Very true, haskell was a bad example, but the point remains the same.

> something other than a bit of a hack.

Generally that might be true, but there are lambdas and the like as well as
'map' in perl and python.  If the consensus is guile, that's fine, and
you're absolutely right about it making sense for a gnu project, it's just
that there are many people who are uncomfortable with, or even militantly
opposed to, lisp based languages.

> it wasn't worth the effort, so that's not much of a boon. :)

Agreed.

> obviously excels at many things that are entirely unrelated to editing,

To clarify, I don't think emacs is necessarily bad at what it's meant for,
but it's easy to fall into that trap when you can do anything in the
program. For that matter it's easy to get into a pattern of adding more
features instead of improving the existing ones, something of which many
projects are guilty but it's best to avoid if possible.

> iD8DBQFIiPEu7M8hyUobTrERAmTKAJ0fEk04Ir9HJCxpzBxH3JFAim+wiwCfd/vm
> OWOFmVvTHGDk9PXhhbeEz8A=
> =el6F

One way or another, I think any way this gets done will put the project
ahead of where it is.  For that matter I'm glad to see so much life back in
this list. =)

---

From: Mark Eichin <[email-masked]>
Date: Thu, 24 Jul 2008 17:54:45 -0400

> "M-x term" within Emacs pretty much fits the bill.

Combined with wicos.el (or one of the more modern versions, escreen I
think?) you get the switch-screens-in-this-window part too.  The thing
you probably don't get is talks-to-serial-ports :-)

---

From: Micah Cowan <[email-masked]>
Date: Thu, 24 Jul 2008 16:09:15 -0700

Johannes Weiner wrote:
> Hi,
> Only a dumb terminal, though :(

Nope. M-x terminal is the dumb terminal. M-x term gives a fully-emulated
terminal ("eterm").

- --
Micah J. Cowan

---

From: "Trent W. Buck" <[email-masked]>
Date: Fri, 25 Jul 2008 11:22:09 +1000

> /Lisp probably has a larger developer base than Haskell does.

Last time I looked, on Freenode, #haskell has around double the number
of people that #scheme has.  While I think more people know *of*
Scheme and Lisp and understand the basics, my impression is that there
is a lot more people actively writing Haskell every day than there are
people actively writing Scheme every day.

However, I wouldn't recommend Haskell as an extension language for an
existing C-based application -- Haskell isn't designed for that role.
(cf. Lua was designed for *exactly* that role.)

> GNU projects (it's a GNU project, so would be "eating our own dogfood").

How many *active* projects use guile as an extension language, and
aren't trying to get rid of it?  How large is the guile user community
(people writing code and libraries in guile)?  How active is the guile
developer community (people improving guile itself)?

My impression is that *nobody* likes Guile.  At all.  Over the years,
I've met *one* guile user who actively advocated it, and about twelve
months ago he learned CL and admitted that he liked that much better.

>>> I would really like to see scripting, but if it means an
>>> emacs-like distribution of 100+ MB of scripting files and the
>>> generation of a program which does everything well except what it
>>> was designed for, then the point has been missed.

$ printf '%s\t%s\n' `grep-aptavail -sInstalled-Size,Package -S --regex ^emacs22 | cut -d: -f 2`
412     emacs22-bin-common
4032    emacs22-common-non-dfsg
7120    emacs22-nox
7548    emacs22
7548    emacs22-gtk
13264   emacs22-el
51780   emacs22-common

emacs22, emacs22-nox and emacs22-gtk are alternative front-ends, so
the smaller two can be ignored in the count.  The emacs22-el package
is not used (since emacs22-common contains the byte-compiled
versions), so it can also be ignored.  Arguably most of emacs22-common
should also be ignored, since it mostly constitutes applications that
are written on top of Emacs and aren't needed by Emacs itself.  Even
if you count it, that's a total of (+ 412 4032 7548 51780) ==> 64MB,
not "100+ MB".

There's no real reason emacs22-common couldn't be split up into the
"core" files needed to run Emacs itself, plus a separate package for
applications.  This isn't done because in general, nobody really cares
about wasting 50MB of disk space.

> To be honest, implementing a Screen within Emacs makes almost as
> much sense as giving Screen Emacs-like scriptability; Screen has
> already duplicated quite a bit of Emacs' layout functionality and
> such, some of it not yet as well as Emacs itself does it. But I
> doubt anyone's interested in seeing that happen. :)

ElScreen is an Emacs window session manager modeled after GNU screen
by NaotoMorishima, http://www.emacswiki.org/cgi-bin/wiki.pl?ElScreen

---

From: "Tom Scogland" <[email-masked]>
Date: Thu, 24 Jul 2008 20:28:45 -0500

> if you count it, that's a total of (+ 412 4032 7548 51780) ==> 64MB,

Comment withdrawn, the number was drawn from the last time I installed
emacs, which was back when I used windows several years ago, and the emacs
package for cygwin was 105mb and the native install was 120mb.  Evidently it
has been streamlined significantly in the meantime.

---

From: Johannes Weiner <[email-masked]>
Date: Thu, 24 Jul 2008 18:11:33 +0200

Hi,

>   cares to write bindings for)

I once hacked something together that was based on guile but I ran into
problems with user input, IIRC.  I will try to look at the problem again
in the near future, hopefully.

I really like the idea.  Thanks for doing this!

By the way, Python is also embeddable :)

> - I would refer to the commit message at [1]

Screw the bloat, there are larger full-blown window managers around :-)

	Hannes

---

From: Johannes Weiner <[email-masked]>
Date: Thu, 24 Jul 2008 23:42:23 +0200

Hi Tom,

"Tom Scogland" <[email-masked]> writes:

> probably be avoided.  Either have it be all scripted, or have it be a C
> core which can be extended and modified from outside, but avoiding a
> situation where most of the program is one way and certain parts are
> another, that will just generate frustration and maintenance problems.

This is just because the Emacs developers chose to include 2+
mailclients and 2+ irc clients into the standard distribution.

That is no must, I think.  I hope that having screen scriptable will
lead to a lot of cool scripts being published and shared, but the screen
distribution can still be kept at a minimum.

	Hannes

---

From: Johannes Weiner <[email-masked]>
Date: Fri, 25 Jul 2008 01:01:10 +0200

Hi,

http://hannes.saeurebad.de/images/no-recursive-emacs.png

	Hannes

---

From: Johannes Weiner <[email-masked]>
Date: Fri, 25 Jul 2008 02:34:11 +0200

Hi,

> Nope. M-x terminal is the dumb terminal. M-x term gives a fully-emulated

Awesome, did not know at all!  Thanks for the hinter!

	Hannes

---

From: Johannes Weiner <[email-masked]>
Date: Fri, 25 Jul 2008 15:57:01 +0200

> you probably don't get is talks-to-serial-ports :-)

M-x serial-term

	Hannes

---

From: "Trent W. Buck" <[email-masked]>
Date: Sun, 27 Jul 2008 03:33:09 +1000

> http://hannes.saeurebad.de/images/no-recursive-emacs.png

M-x term is not the same as M-x shell.  The former provides VT220 (or
thereabouts) emulation; the latter provides dumb terminal emulator (on
top of which is layered the standard Emacs buffer interaction
mechanisms).

Emacs is off-topic for this list, let's all go back to focusing on
Screen.

---

From: "Trent W. Buck" <[email-masked]>
Date: Sun, 27 Jul 2008 03:46:24 +1000

> Screw the bloat, there are larger full-blown window managers around :-)

I don't think that the existence of bloat in *some* solutions for an
unrelated problem domain is an excuse to add bloat to Screen.  Or are
you specifically referring to bloated window managers for the tty?

Note that I don't believe replacing the existing, limited
domain-specific language (DSL) used in ^A: and .screenrc with an
embedded general-purpose language will *necessarily* add bloat.

Using a third-party language would (hopefully) reduce the amount of
code that the Screen developers need to maintain, as well as the
obvious benefit to users of having more customizability.

> language.

I'd certainly like the *core* of Screen to remain in C, or some other
language for which an efficient optimizing compiler exists.  I don't
mind if extra bits (for example, the code to turn the window data into
a string for %w) was converted to a language that's interpreted
on-the-fly.  Separation of "core" from "extra wank that users insist
on" might also reduce the burden on the Screen maintainers.