KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#typescript #ai-agents #cli #open-source #ai #claude-code #codex #developer-tools #react #windows #deepseek-harness #ai-agent

fancyflags

★ 34 Open GitHub ↗

A Python library for defining structured command-line flags.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# fancyflags

![PyPI Python version](https://img.shields.io/pypi/pyversions/fancyflags)
![PyPI version](https://badge.fury.io/py/fancyflags.svg)

## Introduction

`fancyflags` is a Python library that extends
[`absl.flags`](https://github.com/abseil/abseil-py) with additional structured
flag types.

`fancyflags` provides flags corresponding to structures such as
[dicts](#dict-flags), [dataclasses, and (somewhat) arbitrary callables](#auto).

These flags are typed and validated like regular `absl` flags, catching errors
before they propagate into your program. To override values, users can access
familiar "dotted" flag names.

TIP: Already a `fancyflags` user? Check out our [usage tips](#tips)!

## A short note on design philosophy:

`fancyflags` promotes mixing with regular `absl` flags. In many cases a few
regular `absl` flags are all you need!

`fancyflags` does not require you to modify library code: it should only be used
in your "main" file

`fancyflags` is not a dependency injection framework, and avoids
programming-language-like power features. We prefer that users write regular
Python for wiring up their code, because it's explicit, simple to understand,
and allows static analysis tools to identify problems.

## Installation

`fancyflags` can be installed from PyPI using `pip`:

```shell
pip install fancyflags
```

It can also be installed directly from our GitHub repository:

```shell
pip install git+git://github.com/deepmind/fancyflags.git
```

or alternatively by checking out a local copy of our repository and running:

```shell
pip install /path/to/local/fancyflags/
```

## Dict flags

If we have a class `Replay`, with arguments `capacity`, `priority_exponent` and
others, we can define a corresponding dict flag in our main script

```python
import fancyflags as ff

_REPLAY_FLAG = ff.DEFINE_dict(
    "replay",
    capacity=ff.Integer(int(1e6)),
    priority_exponent=ff.Float(0.6),
    importance_sampling_exponent=ff.Float(0.4),
    removal_strategy=ff.Enum("fifo", ["rand", "fifo", "max_value"])
)
```

and `**unpack` the values directly into the `Replay` constructor

```python
replay_lib.Replay(**_REPLAY_FLAG.value)
```

`ff.DEFINE_dict` creates a flag named `replay`, with a default value of

```python
{
    "capacity": 1000000,
    "priority_exponent": 0.6,
    "importance_sampling_exponent": 0.4,
    "removal_strategy": "fifo",
}
```

For each item in the dict, `ff.DEFINE_dict` also generates a dot-delimited
"item" flag that can be overridden from the command line. In this example the
item flags would be

```
replay.capacity
replay.priority_exponent
replay.importance_sampling_exponent
replay.removal_strategy
```

Overriding an item flag from the command line updates the corresponding entry in
the dict flag. The value of the dict flag can be accessed by the return value
of `ff.DEFINE_dict` (`_REPLAY_FLAG.value` in the example above), or via the
`FLAGS.replay` attribute of the `absl.flags` module. For example, the override

```shell
python script_name.py -- --replay.capacity=2000 --replay.removal_strategy=max_value
```

sets `_REPLAY_FLAG.value` to

```python
{
    "capacity": 2000,  # <-- Overridden
    "priority_exponent": 0.6,
    "importance_sampling_exponent": 0.4,
    "removal_strategy": "max_value",  # <-- Overridden
}
```

## Nested dicts

fancyflags also supports nested dictionaries:

```python
_NESTED_REPLAY_FLAG = ff.DEFINE_dict(
    "replay",
    capacity=ff.Integer(int(1e6)),
    exponents=dict(
        priority=ff.Float(0.6),
        importance_sampling=ff.Float(0.4),
    )
)
```

In this example, `_NESTED_REPLAY_FLAG.value` would be

```python
{
    "capacity": 1000000,
    "exponents" : {
      "priority": 0.6,
      "importance_sampling": 0.4,
    }
}
```

and the generated flags would be

```
replay.capacity
replay.exponents.priority
replay.exponents.importance_sampling
```

### Help strings

fancyflags uses the item flag's name as the default help string, however this
can also be set manually:

```python
_NESTED_REPLAY_FLAG = ff.DEFINE_dict(
    "replay",
    capacity=ff.Integer(int(1e6), "Maximum size of replay buffer."),
    exponents=dict(
        priority=ff.Float(0.6),  # Help string: "replay.exponents.priority"
        importance_sampling=ff.Float(0.4, "Importance sampling coefficient."),
    )
)
```

## "Auto" flags for functions and other structures {#auto}

`fancyflags` provides several "auto" functions for generating flags
corresponding to callables, dataclasses and mappings. In these functions, the
type information and default values are inferred from a provided callable or
example structure.

This is more concise than spelling out individual items like `ff.Float(...)`,
and provides stronger static analysis support via typed containers such as
dataclasses.

### `DEFINE_auto`

`ff.DEFINE_auto` automatically generates a flag
declaration corresponding to the _signature_ of a given callable. The return
value will also carry the correct type information.

For example the callable could be a constructor

```python
_REPLAY = ff.DEFINE_auto('replay', replay_lib.Replay)
```

or it could be a container type, such as a `dataclass`

```python
@dataclasses.dataclass
class DataSettings:
  dataset_name: str = 'mnist'
  split: str = 'train'
  batch_size: int = 128

# In main script.
# Exposes flags: --data.dataset_name --data.split and --data.batch_size.
_DATA_SETTINGS = ff.DEFINE_auto('data', datasets.DataSettings)

def main(argv):
  # del argv  # Unused.
  dataset = datasets.load(_DATA_SETTINGS.value())
  # ...
```

or any other callable that satisfies the `ff.auto` requirements. It's also
possible to override keyword arguments in the call to `.value()`, e.g.

```python
test_settings = _DATA_SETTINGS.value(split='test')
```

#### Parser overrides

Both `ff.DEFINE_auto` and `ff.DEFINE_from_instance` accept an optional
`parser_overrides` argument, which is a mapping from a parameter name to a
`ff.Item` or `ff.MultiItem` instance. This is useful when the default flag
parsing isn't desired. For example, to use `ff.TypedList` for a sequence
instead of the default parser:

```python
@dataclasses.dataclass
class DataSettings:
  dataset_name: str = 'mnist'
  batch_size: int = 128
  augmentations: Sequence[str] = ('flip', 'crop')

_DATA_SETTINGS = ff.DEFINE_auto(
    'data',
    DataSettings,
    parser_overrides={
        'augmentations': ff.TypedList(str, ('flip', 'crop'), 'augmentations'),
    },
)
```

This allows `--data.augmentations=flip,crop,rotate` on the command line,
instead of the default parser which expects
`--data.augmentations=[flip,crop,rotate]`.

Dotted keys are supported for overriding fields within nested dataclasses:

```python
@dataclasses.dataclass
class Augmentation:
  transforms: Sequence[str] = ('flip', 'crop')
  probability: float = 0.5

@dataclasses.dataclass
class DataSettings:
  dataset_name: str = 'mnist'
  augmentation: Augmentation = dataclasses.field(default_factory=Augmentation)

_DATA_SETTINGS = ff.DEFINE_auto(
    'data',
    DataSettings,
    parser_overrides={
        'augmentation.transforms': ff.TypedList(
            str, ('flip', 'crop'), 'transforms'
        ),
    },
)
```

This allows `--data.augmentation.transforms=flip,crop,rotate` on the command
line.

### `DEFINE_from_instance`

`ff.DEFINE_from_instance` automatically generates a flag declaration
corresponding to the _value_ of a given instance of a dataclass or mapping. As
with `ff.DEFINE_auto`, the return value will also carry the correct type
information.

`ff.DEFINE_from_instance` supports nested structures, as in the example below,
and is useful for defining flags which contain different values, without needing
to rewrite or subclass library code:

```python
@dataclasses.dataclass
class DataSettings:
  dataset_name: str = 'mnist'
  split: str = 'train'
  batch_size: int = 128

@dataclasses.dataclass
class ExperimentSettings:
  data: DataSettings
  ...

# In main script.
# Exposes flags: --config.data.dataset_name, etc
_SETTINGS = ff.DEFINE_from_instance(
    # It's easy to choose a different value for the provided instance.
    'config', ExperimentSettings(data=DataSettings(batch_size=64))
)
```

Materialising a value works the same as for `ff.DEFINE_auto`:

```python
settings = _SETTINGS.value()
```

`ff.DEFINE_from_instance` also supports `parser_overrides` in the same way as
`ff.DEFINE_auto` (see [parser overrides](#parser-overrides) above).

## Defining a dict flag from a function or constructor.

The function `ff.auto` returns a dictionary of `ff.Items` given a function or
constructor. This is used to build `ff.DEFINE_dict` and is also exposed in the
top-level API.

`ff.auto` can be used with `ff.DEFINE_dict` as follows:

```python
_WRITER_KWARGS = ff.DEFINE_dict('writer', **ff.auto(logging.Writer))
```
`ff.auto` may be useful for creating kwarg dictionaries in situations where
`ff.DEFINE_auto` is not suitable, for example to pass kwargs into nested
function calls.

## Auto requirements

`ff.DEFINE_auto` and `ff.auto` will work if:

1.  The function or class constructor has type annotations.
1.  The types of the arguments are relatively simple types (`int`, `str`,
    `bool`, `float`, or sequences thereof).

## Notes on using `flagsaver`

abseil-py's [flagsaver](https://github.com/abseil/abseil-py/blob/master/absl/testing/flagsaver.py)
module is useful for safely overriding flag values in test code. Here's how to
make it work well with fancyflags.

### Making dotted names work with `flagsaver` keyword arguments

Since `flagsaver` relies on keyword arguments, overriding a flag with a dot in
its name will result in a `SyntaxError`:

```python
# Invalid Python syntax.
flagsaver.flagsaver(replay.capacity=100, replay.priority_exponent=0.5)
```

To work around this, first create a dictionary and then `**` unpack it:

```python
# Valid Python syntax.
flagsaver.flagsaver(**{'replay.capacity': 100, 'replay.priority_exponent': 0.5})
```

### Be careful when setting flag values inside a `flagsaver` context

If possible we recommend that you avoid setting the flag values inside the
context altogether, and instead pass the override values directly to the
`flagsaver` function as shown above. However, if you _do_ need to set values
inside the context, be aware of this gotcha:

This syntax does not work properly:

```python
with flagsaver.flagsaver():
  FLAGS.replay["capacity"] = 100
# The original value will not be restored correctly.
```

This syntax _does_ work properly:

```python
with flagsaver.flagsaver():
  FLAGS["replay.capacity"].value = 100
# The original value *will* be restored correctly.
```

## fancyflags in more detail

### What is an `ff.Float` or `ff.Integer`?

`ff.Float` and `ff.Integer` are both `ff.Item`s. An `ff.Item` is essentially a
mapping from a default value and a help string, to a specific type of flag.

The `ff.DEFINE_dict` function traverses its keyword arguments (and any nested
dicts) to determine the name of each flag. It calls the `.define()` method of
each `ff.Item`, passing it the name information, and the `ff.Item` then defines
the appropriate dot-delimited flag.

### What `ff.Item`s are available?

### Defining a new `ff.Item`

Given a `flags.ArgumentParser`, we can define an `ff.Item` in a few lines of
code.

For example, if we wanted to define an `ff.Item` that corresponded to
`flags.DEFINE_spaceseplist`, we would look for the parser that this definition
uses, and write:

```python
class SpaceSepList(ff.Item):

def __init__(self, default, help_string)
    parser = flags.WhitespaceSeparatedListParser()
    super(SpaceSepList, self).__init__(default, help_string, parser)

```

Note that custom `ff.Item` definitions do not _need_ to be added to the
fancyflags library to work.

### Defining `Item` flags only

We also expose a `define_flags` function, which defines flags from a flat or
nested dictionary that maps names to `ff.Item`s. This function is used as part
of `ff.DEFINE_dict` and `ff.DEFINE_auto`, and may be useful for writing
extensions on top of `fancyflags`.

```python
_writer_items = dict(
    path=ff.String('/path/to/logdir', "Output directory."),
    log_every_n=ff.Integer(100, "Number of calls between writes to disk."),
)

_WRITER_KWARGS = ff.define_flags("writer", _writer_items)
```

This example defines the flags `writer.path` and `writer.log_every_n`
only: does _not_ define a dict-flag. The return value (`_WRITER_KWARGS`) is a
dictionary that contains the default values. Any overrides to the individual
flags will also update the corresponding item in this dictionary.

### Tips

Any direct access, e.g. `_DICT_FLAG.value['item']` is an indication that you
may want to change your flag structure:

*   Try to align dict flags with constructors or functions, so that you always
    `**unpack` the items into their corresponding constructor or function.
*   If you need to access an item in a dict directly, e.g. because its value is
    used in multiple places, consider moving that item to its own plain flag.
*   Check to see if you should have `**unpacked` somewhere up the call-stack,
    and convert function "config" args to individual items if needed.
*   Don't group things under a dict flag just because they're thematically
    related, and don't have one catch-all dict flag. Instead, define individual
    dict flags to match the constructor or function calls as needed.

fancyflags

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README