buildsomething/content/blog/entry1.md

+++
title = "arara version 6 released"
description = "arara version 6 released"
date = 2024-11-05
+++

After approximately one year, we are proud to announce the release of arara
version 6. Just ready for TeX Live 2021, we have finished many exciting new
features and are already prepared to work on a technically even more intriguing
version 7.

<!-- more -->

# Important changes

Please consult our [changelog](google.com) for a detailed view on all
changes and references to our issue tracker. This post which is targeted at our
users will only cover the most relevant changes.

## More configurable and default preambles

Preambles are the user's way of not writing a rule yet reusing certain
directives. Up to now, preambles have been prepended to the file regardless of
it containing other directives. That means, if you have the following in your
configuration:

```yaml
preambles:
  twopdflatex: |
    % arara: pdflatex
    % arara: pdflatex
```

and your `file.tex` contains

```latex
% arara: lualatex
```

you might actually want to compile with `lualatex`. In version 5 and before, a
call to `arara -p twopdflatex file.tex` would result in two compilations using
`pdflatex` and one using `lualatex`.

If you do not want `arara` to perform these preamble compilations if there are
directives present in your file, you may now set the following option in your
`arararc` file:

```yaml
prependPreambleIfDirectivesGiven: false
```

The default is still `true` to preserve backward compatibility.

Another major improvement for preambles is the ability to specify a
`defaultPreamble`, i.e. you may now even omit `-p preamble` if you specified
that key in your configuration file. This is especially useful if all files in
your project shall be run with the same configuration. You may then set the
following in your `arararc`:

```yaml
preambles:
  twopdflatex: |
    % arara: pdflatex
    % arara: pdflatex
defaultPreamble: twopdflatex
```

and can call `arara file.tex file2.tex file3.tex` where arara will automatically
run the default preamble on each of these files.

Combining those features is even more intriguing. If you insert a snippet like

```yaml
preambles:
  twopdflatex: |
    % arara: pdflatex
    % arara: pdflatex
defaultPreamble: twopdflatex
prependPreambleIfDirectivesGiven: false
```

into your *global* arara configuration file, you can run `arara file.tex` on any
file around there and it will either

* use the directives from the file if given or
* resort to the default preamble.

That way, arara can be set as the default compiler in your editor and you can
even compile before having inserted any arara directive.

> Important note: Of course, this destroys the portability arara directives use
> to create. Please do not get lazy and specify directives in documents you want
> to share.

## Passing command-line parameters in arara's evaluation

In previous versions, the only way to communicate with arara in rules and (with
some trickery) directives were environment variables (as of version 5) and file
I/O (since the beginning). This made automated builds only configurable with
writing own rules which some users tend to avoid for simple directive patterns.

In version 6, we introduce a new command line parameter `-P` to allow specifying
  command-line options to be passed to arara's session map. Calling arara like

```sh
$ arara -P key=value file.tex
```

will allow you to write the following directives in `file.tex`:

```latex
% arara: pdflatex
% arara: pdflatex if (getSession().get('arg:key') == 'value')
```

Of course this is a poor example. But you get the gist of what is now possible
using this command-line parameter. Imagine writing preambles in your
`arararc.yaml` file that perform tests like `getSession().get('arg:run') ==
'fast')` to leave certain compilations out for fast runs.


## Expansion within `options`

The new command-line parameters are one of the few commands that have been made
available within the `options` part of directives. This is especially useful to
set output names or job names like

```latex
% arara: pdflatex: { options: ["-jobname=@{getSession().get("arg:key")}"] }
```

Apart from `getSession` you may use `getBasename` and `getOriginalReference`
with their documented meanings in the `options` expansion.

## A new safe mode

For quite some time, arara has been a very powerful tool. With some of the new
changes, e.g. letting the user specify a full path anywhere on the file system
to place the log file at, the wishes for a safe mode seemed very sensible.

The new safe mode may be enabled using the command-line flags `-S` or
`--safe-run`. Currently, the following features are restricted:

* file lookup: Where arara called as `arara file` will usually try sensible
  extensions, in safe mode arara only does what it is asked to do which means
  only operating on files as given. It won't find `file.tex` in safe mode which
  would be operated on in a normal run.
* `unsafelyExecuteSystemCommand` will raise an exception and abort the run
  (although rules are still allowed to construct arbitrary commands using
  a `return getCommand(…)` approach so this only disallows arbitrary system
  commands that would not get logged and are thus invisible to the user)
* the `options` parameter does not expand orb tags in any directive.

## Improving safety of flags and defaults in rules

In rules, you specify `flag`s and `default`s for parameters. We changed the
handling of both. Let's start with `default`. Up to now, `default` could compute
expensive operations and in many cases duplicated lots of code from the
corresponding `flag`. Because of that, arara changed behavior and now treats the
`default` like user input. To make that more clear, let's consider the example
directive

```latex
% arara: pdflatex: {branch: stable}
```

where the `stable` branch is actually the default value. In previous versions,
you would write (in your rule)

```yaml
arguments:
- identifier: branch
  flag: >
    @{
        return [ 'stable' : 'pdflatex', 'developer' : 'pdflatex-dev' ]
                 .get(parameters.branch);
    }
  default: 'pdflatex'
```

which expresses that you want to run the `stable` branch by default but only
because the rule author knows that if the user uses `stable` the `flag` will
return `pdflatex`.

To express the same thing in v6, i.e. to use the stable branch of this rule by
default, you would now write

```yaml
arguments:
- identifier: branch
  flag: >
    @{
        return [ 'stable' : 'pdflatex', 'developer' : 'pdflatex-dev' ]
                 .get(parameters.branch);
    }
  default: 'stable'
```

which is very clear, even to the user who only knows the documented options of
the rule. Furthermore, it now expresses the intention. As with user input, the
`flag` will be called on the default value and determine the real value.

This change also implies that there is no orb tag expansion in the `default`
field anymore.

The other notable change for flags is the transformation into a
  `List<String>`. That means for the return types of a `flag`:

* If a `flag` returns a `List`, it will be flattened and all values will be
  turned into string.
* If a string is returned, a single list with only that string will be
  returned.
* Other values (which are explicitly discouraged) are transformed into a
  string.

Consider the argument example above: This `branch` argument actually returns a
`String`, e.g. `pdflatex`. Hence, it is part of the second category. Internally,
arara transforms this return value into `['pdflatex']`, i.e. a list with one
single element.

If you need interoperability of complex `command` or `flag` code with older
  versions, use `isList(variable) ? variable[0] : variable` to get the value of
  previously non-List values.

## Rule updates

Some rules have been updated or added. New rules include `ghostscript`, `copy`,
  `move`, `ltx2any`, `llmk`, `spix`, `pdfcrop` and `sage`. If you encounter any
  issues, please open an issue at GitLab. We are also always open to support
  more tools in future versions.

> Important note on rules: Those rules included in the core distribution have
> been renamed to have a unique prefix in the texmf tree. File names should not
> be relied upon (see deprecation notice in v5).

## Being forgiving about language choices

Earlier last year, we started a poll about arara's localization and the users we
have got feedback from stated they would be good with arara being focused on the
English user interface. In version 5 we already removed the localized
command-line interface/help, even without knowing about user feedback.

For this version, we decided to let arara continue running in English if an
invalid language is specified. Previously, specifying a language like `quack`
would cause arara to end the execution which does not seem appropriate anymore.

Another related change has been the switch to IETF BCP 47 codes for language
selection. Please refer to the [changelog](google.com) for details.

## Removal of the arara shorthand

As announced, the (as of version 5.0.0) deprecated `<arara>` shorthand notation
  has been removed. This is a breaking change. If your document previously used
  rules with

```yaml
flag: <arara> return ['a', 'b'];
```

you will now have to write the more elaborate but much clearer

```yaml
flag: >
  @{
    return ['a', 'b'];
  }
```

## Fixed some minor issues

In arara version 5 and even before, the exit status handling did not conform to
  the documentation. We only differentiate the exit values 0, 1 and 2 as valid
  exit values for arara. Prior to this fix you have gotten the exit code of the
  failing command directly. For scripting purposes this could be a nice
  enhancement but we decided to aim for reestablishing documented behavior.

The correct behavior of `currentFile` and `getOriginalReference` has been
  restored as version 5 incorrectly used the same object for both.

# The next steps

For version 7 of arara, i.e. TeX Live 2022 we have big plans. Not all of them
may eventually end up in the released version but we still want to share our
ideas with you.

## Trying to serve more platforms

arara is written in Kotlin, a very versatile language. It has been designed with
compilation to different platforms in mind. The usual base of operations for
Kotlin is the JVM and the Java ecosystem. But one may also compile Kotlin to
JavaScript or even native binaries.

In version 6, we already made a split between an API, a core implementation, the
MVEL rules and the CLI application to separate concerns. This was the first step
in the direction of splitting out components that are bound to one
platform. Aditionally, we hope to enable other people to develop their own tools
based on the arara libraries if they wish to. With multiple library components
this should be easy.

For version 7, we want to take larger steps towards
platform-independence. Hence, we will make use of Kotlin/Multiplatform (MPP) to
use a single codebase for all target platforms. Some components will have to be
rewritten, some need different interfaces for different platforms but in the
end, we hope to provide an even more multitalented tool.

The real challenge is a major change of the API so that we are able to interpret
rules that have been written for MVEL (which is JVM-only) to work on native
platforms (Windows/Linux/Mac binaries) with respective libraries. We will have
to put quite some effort into improving our API.

Last but not least, we want to drop some word about the background of these
plans: arara has never known to be the fastest player among the TeX build tools
but nevertheless performance is important to us. That, combined with some
complaints (e.g. by a big player in the TeX world) about the reliance on the
JVM, made us curious where we can take arara in that direction.

## Exploring new grounds

Up to now, arara has been developed as a file-based build tool. The tool does
not track dependencies nor does it manage projects. You throw a file at it and
it makes the best out of it. That results in directives like the following
snippet from `file.tex`

```latex
% arara: metapost: { files: [a.mp, b.mp] }
% arara: pdflatex
```

which in fact does not (only) specify how to compile `file.tex` but it specifies
three things:

1. how to compile `a.mp`,
2. how to compile `b.mp` and
3. how to compile `file.tex`.

In fact, we are doing project management by hand. Obviously, the author intended
to compile the metapost files before running LaTeX. That is a clear indication
of dependency management.

As we have been asked about `make`-like support of directive specifications out
of the source file, we think the only sensible approach to tackle it is to
actually introduce the concept of a project to kill two birds with one stone.

Actually, we already worked on project support in combination with a new
Kotlin-based DSL (domain-specific language) for version 6. To avoid major
problems, we decided not to ship it and reconsider our plans. However, we
already had project specifications like

```kts
project("My awesome book") {
  file("a.mp") {
    directives = listOf("metapost")
  }
  file("b.mp") {
    directives = listOf("metapost")
  }
  file("file.tex") {
    dependsOn("a.mp", "b.mp")
    // directives omitted, take from file
  }
}
```

available, so stay tuned.