Development

Rationale

The LuaTeX project has several properties that result in the objectives and organization mentioned below.

  1. It started out as a wild idea and experiment but by now it is a real project in which we find much joy. We want to keep it that way.

  2. The project originates from the ConTeXt community and it was a grant from one of its users that gave the project a jump start and the perspective of realizing ambitious goals in a reasonable time frame.

  3. We feel that by following the approach mentioned we can open up TeX and bring users the control they want. The project is a logical follow up of the pdfTeX project. With this project it shares the property of controlled development by a small team.

Objectives

The project has the following objectives. We may add more if we feel the need. The objectives are reasonable and we are confident that we can fulfil them.

  1. The project will provide a stable version of TeX enhanced with the Lua scripting engine. This will permit users to do basic text manipulations and runtime calculations.

  2. The project will merge pdfTeX and Aleph (Omega) so that it can be used as a replacement. Using the Lua features is not needed in order to perform as expected. The project team will provide feature downward compatibility where possible.

  3. The project will drop (non TeX related) features from the code base that are better implemented using standard Lua code. No input reencoding will be implemented (can be done via overloads) and internal data path is utf based. Some pdfTeX features like timing, expressions, recoding and such will be dropped because it increases the code base as well as makes it dependent on libraries.

  4. The project will open up TeX's internals to a Lua interface. This will be done by providing hooks to all relevant places in the flow from input to output. In addition internal variables will be accessable.

  5. The project will extend the font interface so that users can adapt font support to their needs. Readers will be provided the binary formats tfm (TeX Font Metrics), otf and ttf (OpenType). A minimum set of helper functions will be implemented.

  6. The project will permit users to overload file lookup, reading from files, converting lines of input, manipulating tokens, manipulating node lists and enhancing the output stream. This will provide much control and permit advanced typography.

  7. The project will clean up various parts of the code base and the compilation machinery. A few additional some functions will be added to the standard Lua repertoire. LuaTeX can act as a stand alone Lua interpreter.

  8. The project will interface to the MetaPost library that is developed in a subproject.

  9. The project will provide sample code as part of the ConTeXt distribution. In that distribution there is also generic code suitable for plain TeX.

Organization

TeX has been stable for over 25 years (speaking in 2007) and development was concentrated around a few people. With LuaTeX we follow the same approach.

  1. The core team is small and will be kept small: Hans, Hartmut and Taco. They are responsible for interfaces, coding, and quality assurance. Stable snapshots will be made available periodically but development version are to be taken as they are. New features and experimental code may come and go in short cycle.

  2. Taco, Idris and Hans are also members of the Oriental TeX project. This project funds part of the LuaTeX code development. It aims at typesetting high quality arab. This project also acts as an example or how generic interfaces can be used for specific solutions. This project roots development of LuaTeX in the academic world.

  3. The project is an open source project in the literal sense. One can look at the source code, take the code, spawn off a branch, make private versions, etc. However, since it's the core team's brain child, it's op to the core team to determine the directions. TeX as well as Lua evolved in a controlled way and as a result they are stable and/or have well defined transition stages. LuaTeX will evolve in a similar way: controlled. As with TeX and Lua it's the interfaces that provides the users with a way to implement their needs.

  4. Once the first version is available, part of the maintaince will be done by the pdfTeX team members (release management, bug fixes) in their usual roles.

  5. The code repository is readable by anyone but new code and patches are the responsibility of the core team. For TeX distributions stability is a must. Bug fixes will be applied when they have undergone proper quality assurance.

  6. Only features documented as stable will remain as they are. If users want extra features they can ask for it in which case it will be discussed in the team. Keep in mind that wit LuaTeX we provide interfaces and glue, no solutions. Because we expect macro packages and users to have diverse and even conflicting demands, we stick to low level interfaces.

  7. Technically users can develop additional libraries (written in Lua) or in any language that can glue to Lua. These are not the responsibility of the team.

  8. The team uses ConTeXt MkIV as testbed. They have control over the development of this macro package which permits them to change code frequently and throw away solutions, demand recoding of large parts, etc.