KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #codex #typescript #developer-tools #ai-agents #deepseek-harness #claude-code #ai-agent #cli #react #mcp

fjta

★ 112 Open GitHub ↗

FJTA (Forensic Journal Timeline Analyzer) is a tool that analyzes Linux filesystem (ext4, XFS) journals (not systemd-journald logs), generates timelines, and detects suspicious activities.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# FJTA - Forensic Journal Timeline Analyzer

FJTA (Forensic Journal Timeline Analyzer) is a tool that analyzes Linux filesystem (ext4, XFS) journals (not systemd-journald logs), generates timelines, and detects suspicious activities.

> [!CAUTION]
> Since testing is only being done with simple disk images, there may be many issues when analyzing more practical disk images.

## Features

- **Journal Analysis**: Scans ext4 and XFS journals to visualize modification history.
- **Timeline Generation**: Organizes events within the journal in chronological order.
- **Suspicious Activity Detection**: Identifies deleted files and potentially tampered operations.
- **Cross-Platform**: Written in Python, allowing analysis on any operating system.

## Supported Artifacts in Filesystem Journals

| Artifacts                              |  ext4  | XFS |
| -------------------------------------- | :----: | :-: |
| inode                                  |   ✅   | ✅  |
| Directories with few entries           |   ✅   | ✅  |
| Directories with many entries          | ✅[^1] | ✅  |
| Short symlink target names             |   ✅   | ✅  |
| Long symlink target names[^2]          |   ✅   | ❌  |
| Short extended attributes              |   ✅   | ✅  |
| Long extended attributes[^3]           | ✅[^4] | ❌  |
| Non-regular files (e.g. block devices) |   ✅   | ✅  |
| Year 2038 problem                      |   ✅   | ✅  |
| Exported journals                      |   ✅   | ✅  |

[^1]: Currently, only linear directories can be parsed. Support for hash tree directories will be added in future versions.

[^2]: Symlink target names stored outside an inode.

[^3]: Extended attributes stored outside an inode.

[^4]: Only the first data block assigned to the extended attribute is recognized. The EXT4_FEATURE_INCOMPAT_EA_INODE flag is not supported.

## Detectable Activities

| Activities                            | ext4 | XFS |
| ------------------------------------- | :--: | :-: |
| Creating files                        |  ✅  | ✅  |
| Deleting files                        |  ✅  | ✅  |
| Modification of extended attributes   |  ✅  | ✅  |
| Timestomping (timestamp manipulation) |  ✅  | ✅  |
| Other inode metadata changes[^5]      |  ✅  | ✅  |

[^5]: "Other inode metadata changes" include updates to MACB timestamps (mtime, atime, ctime, and crtime), file size changes, and setting file flags, and more.

## Requirements

Tested with the following software and libraries:

- [Python](https://www.python.org/) 3.12
- [The Sleuth Kit](https://github.com/sleuthkit/sleuthkit) 4.14.0
- [pytsk3](https://github.com/py4n6/pytsk) 20250729
- [Construct](https://github.com/construct/construct) 2.10.70
- [python-magic](https://github.com/ahupp/python-magic) 0.4.27
- [libewf-python](https://pypi.org/project/libewf-python/) 20240506
- [libvmdk-python](https://pypi.org/project/libvmdk-python/) 20240510
- [libvhdi-python](https://pypi.org/project/libvhdi-python/) 20240509
- [yaspin](https://github.com/pavdmyt/yaspin) 3.4.0

## Installation From Source

Compile and install the TSK.

> [!NOTE]
> TSK also requires other libraries such as libewf, libvmdk, and so on.

```bash
wget https://github.com/sleuthkit/sleuthkit/releases/download/sleuthkit-4.14.0/sleuthkit-4.14.0.tar.gz
tar xvzf sleuthkit-4.14.0.tar.gz
cd sleuthkit-4.14.0
./configure
make
sudo make install
sudo echo /usr/local/lib > /etc/ld.so.conf.d/local-lib.conf
sudo ldconfig
```

Then, clone FJTA.

```bash
git clone https://github.com/mnrkbys/fjta.git
cd fjta
```

Finally, install required Python packages.

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install pytsk3 construct python-magic libewf-python libvmdk-python libvhdi-python yaspin
```

## Installation From Packages

Install the TSK package from the Linux distribution you are using.

> [!NOTE]
> In older versions of libvmdk, you cannot open VMDK files created with VMware Workstation for Windows (Japanese edition).
> The patch was integrated in [2022](https://github.com/libyal/libvmdk/pull/31).

```bash
sudo apt install sleuthkit python3-tsk libewf2 libvmdk1 libvhdi1 python3-libewf python3-libvmdk python3-libvhdi
```

Then, clone FJTA.

```bash
git clone https://github.com/mnrkbys/fjta.git
cd fjta
```

Finally, install required Python packages.

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install construct python-magic yaspin
```

## Usage

### Basic

```bash
python ./fjta.py -i ~/ext4.img | jq
```

### Save timeline to a file

```bash
python ./fjta.py -i ~/ext4.img --output timeline.ndjson
```

When `--output` is specified, timeline-generation progress spinners are shown by default.
Use `--no-progress` to hide them.

### Filtering with an inode number

```bash
python ./fjta.py -s 0 -i ~/xfs.img | jq 'select(.inode == 101040435)' | less
```

### Filtering with crtime

The to_epoch() function is defined in the helper.sh file, so you need to import it before executing the following command.

```bash
source scripts/helper.sh
python ./fjta.py -s 0 -i ~/xfs.img | jq --argjson threshold $(to_epoch "2025-06-23 07:33:20.123456789") 'select(.crtime >= $threshold)'
```

### Filtering with a filename

```bash
python ./fjta.py -s 0 -i ~/xfs.img | jq 'select(.names? and ([.names[][]] | index("backdoor.c")))'
```

### Filtering with a string

```bash
python ./fjta.py -s 0 -i ~/xfs.img | jq 'select(.info | contains("Added EA: security.selinux"))'
```

### Filtering with a regex pattern

```bash
python ./fjta.py -s 0 -i ~/xfs.img | jq 'select(.info | test("added ea: security\\.selinux"; "i"))'
```

## Sample Output (timestomping)

```bash
...
{
  "transaction_id": 3,
  "action": "CREATE_INODE|CREATE_HARDLINK",
  "inode": 12,
  "file_type": "REGULAR_FILE",
  "names": {
    "2": [
      "test.txt"
    ]
  },
  "mode": 420,
  "uid": 0,
  "gid": 0,
  "size": 0,
  "atime": 1729038807.9101748,
  "ctime": 1729038807.9101748,
  "mtime": 1729038807.9101748,
  "crtime": 1729038807.9101748,
  "dtime": 0.0,
  "flags": 524288,
  "link_count": 1,
  "symlink_target": "",
  "extended_attributes": [],
  "device_number": {
    "major": 0,
    "minor": 0
  },
  "info": "Crtime: 2024-10-16 00:33:27.910174879 UTC|Link Count: 1"
}
...
{
  "transaction_id": 23,
  "action": "CREATE_INODE|ACCESS|CHANGE|MODIFY|TIMESTOMP",
  "inode": 12,
  "file_type": "REGULAR_FILE",
  "names": {
    "2": [
      "test.txt"
    ]
  },
  "mode": 420,
  "uid": 0,
  "gid": 0,
  "size": 0,
  "atime": 978312225.8287878,
  "ctime": 978312225.8287878,
  "mtime": 978312225.8287878,
  "crtime": 978312225.8287878,
  "dtime": 0.0,
  "flags": 524288,
  "link_count": 1,
  "symlink_target": "",
  "extended_attributes": [],
  "device_number": {
    "major": 0,
    "minor": 0
  },
  "info": "Atime: 2024-10-18 08:25:51.385837319 UTC -> 2001-01-01 01:23:45.828787850 UTC (Timestomp)|Ctime: 2024-10-18 08:25:51.385837319 UTC -> 2001-01-01 01:23:45.828787850 UTC (Timestomp)|Mtime: 2024-10-18 08:25:51.385837319 UTC -> 2001-01-01 01:23:45.828787850 UTC (Timestomp)|Crtime: 2024-10-16 00:33:27.910174879 UTC -> 2001-01-01 01:23:45.828787850 UTC (Timestomp)"
}
...
```

## Output Contract (JSON Lines)

FJTA writes one JSON object per line (JSONL/NDJSON) to stdout.

### Event object

Each line is a timeline event with the following fields.

- `transaction_id` (integer): Transaction identifier in journal order.
- `action` (string): One or more action flags joined by `|`.
- `inode` (integer): Inode number.
- `file_type` (string): `REGULAR_FILE`, `DIRECTORY`, `SYMBOLIC_LINK`, etc.
- `names` (object): Directory-inode-to-name-list mapping. Example: `{"2": ["test.txt"]}`.
- `mode` (integer): File mode in decimal.
- `uid`, `gid` (integer): Owner and group IDs.
- `size` (integer): File size in bytes.
- `atime`, `ctime`, `mtime`, `crtime`, `dtime` (number): Epoch timestamps with sub-second precision.
- `flags` (integer): Filesystem-specific flag bitmask.
- `link_count` (integer): Hard-link count.
- `symlink_target` (string): Symlink target if available.
- `extended_attributes` (array): List of `{name, value}` objects.
- `device_number` (object): `{major, minor}` for device files.
- `info` (string): Human-readable details for forensic interpretation.

### `action` semantics

- `action` is a bitflag representation serialized as names joined by `|`.
- Token order is not guaranteed. Parse by splitting on `|` and matching tokens.
- Common values include `CREATE_INODE`, `DELETE_INODE`, `CREATE_HARDLINK`, `DELETE_HARDLINK`, `ACCESS`, `CHANGE`, `MODIFY`, `TIMESTOMP`, `MOVE`, and metadata-change actions.

### Compatibility policy

- Existing documented keys are intended to remain available.
- New keys may be added in future versions.
- Users and downstream tools should ignore unknown keys for forward compatibility.
- The text format of `info` is informative and may change across versions.

### Diagnostics contract

- Timeline events are written to `stdout` only.
- Warnings, debug logs, and parser diagnostics are written to `stderr`.
- `stderr` output is operator-oriented and not a stable machine-parseable API.

## Development and Operations Notes

### Parser boundaries (for contributors)

- `JournalParser.parse_journal()` is the ingestion/parsing phase and must not emit timeline JSON.
- `JournalParserCommon.infer_timeline_events()` builds timeline event objects.
- `JournalParserCommon.emit_timeline_events()` is the only phase that serializes events to JSON Lines on `stdout`.
- `JournalParserCommon.timeline()` orchestrates infer/emit; keep business logic in infer and output logic in emit.

### Running tests locally

- Activate the virtual environment first.
- Run tests with `python -m pytest` (recommended), not bare `pytest`.
- This ensures tests run with the same interpreter/environment as FJTA and avoids module resolution differences.

```bash
source .venv/bin/activate
python -m pytest -q
```

### Performance diagnostics (for operators)

- Enable with `--debug`.
- Performance logs are emitted to `stderr` with `PERF ...` prefixes.
- Current stage logs include:
  - `PERF parse_journal: <seconds>s`
  - `PERF timeline.infer: <seconds>s; count=<events>`
  - `PERF timeline.emit: <seconds>s; count=<events>`
  - `PERF timeline.total: <seconds>s; count=<events>`
- These diagnostics are for observation and tuning; they are not a stable API contract.

## Detecting data exfiltration by filtering and formatting with jq, awk, and column

In the following output example, you can also see a list of the exfiltrated files.

```bash
$ python ./fjta.py -i xfs_data_exfiltration.img | jq -r '
  select(
    (.action == "ACCESS")
    or (.names | to_entries | any(.value[] | test("\\.(zip|rar|7z|gz|bz2)$"; "i")))
  )
  | [
      .inode,
      (.names | tostring),
      .size,
      .action,
      .mode,
      (.mtime  | strftime("%Y-%m-%d %H:%M:%S")),
      (.atime  | strftime("%Y-%m-%d %H:%M:%S")),
      (.ctime  | strftime("%Y-%m-%d %H:%M:%S")),
      (.crtime | strftime("%Y-%m-%d %H:%M:%S"))
    ]
  | @tsv
' |
awk -F'\t' '{printf "%s\t%s\t%s\t%s\t%04o\t%s\t%s\t%s\t%s\n", $1, $2, $3, $4, $5, $6, $7, $8, $9}' | column -s $'\t' -t -N inode,names,size,action,mode,mtime,atime,ctime,crtime
inode    names                       size       action                        mode  mtime                atime                ctime                crtime
132      {"128":["dummy_data"]}      30         ACCESS                        0755  2025-12-19 01:54:10  2025-12-19 01:55:20  2025-12-19 01:54:10  2025-12-19 01:54:10
524416   {"132":["dir1"]}            66         ACCESS                        0755  2025-12-19 01:54:10  2025-12-19 01:55:20  2025-12-19 01:54:10  2025-12-19 01:54:10
1179776  {"524416":["dir2"]}         137        ACCESS                        0755  2025-12-19 01:54:10  2025-12-19 01:55:20  2025-12-19 01:54:10  2025-12-19 01:54:10
1572992  {"524416":["dir3"]}         146        ACCESS                        0755  2025-12-19 01:54:10  2025-12-19 01:55:20  2025-12-19 01:54:10  2025-12-19 01:54:10
...
1179777  {"1179776":["file1"]}       1048576    ACCESS                        0644  2025-12-19 01:54:10  2025-12-19 01:55:20  2025-12-19 01:54:10  2025-12-19 01:54:10
1179778  {"1179776":["file2"]}       1048576    ACCESS                        0644  2025-12-19 01:54:10  2025-12-19 01:55:20  2025-12-19 01:54:10  2025-12-19 01:54:10
1179779  {"1179776":["file3"]}       1048576    ACCESS                        0644  2025-12-19 01:54:10  2025-12-19 01:55:20  2025-12-19 01:54:10  2025-12-19 01:54:10
...
164      {"155":["file99"]}          1048576    ACCESS                        0644  2025-12-19 01:54:11  2025-12-19 01:55:22  2025-12-19 01:54:11  2025-12-19 01:54:11
165      {"155":["file100"]}         1048576    ACCESS                        0644  2025-12-19 01:54:11  2025-12-19 01:55:22  2025-12-19 01:54:11  2025-12-19 01:54:11
167      {"128":["takeout.zip"]}     0          CREATE_INODE|CREATE_HARDLINK  0644  2025-12-19 01:55:22  2025-12-19 01:55:20  2025-12-19 01:55:22  2025-12-19 01:55:20
167      {"128":["takeout.zip"]}     104894079  SIZE_UP                       0644  2025-12-19 01:55:22  2025-12-19 01:55:20  2025-12-19 01:55:22  2025-12-19 01:55:20
```

## How to export filesystem journals

FJTA can analyze exported journals. However, some parameters required for analysis are not included in the journal itself. Therefore, you must also export the corresponding superblock (or filesystem metadata) information.

When exporting both files, make sure they share the same filename without the extension.

Use the following filename conventions:

- \*.journal (or any extension of your choice) for the exported journal
- \*.dumpe2fs for the dumpe2fs output (ext4)
- \*.xfs_info for the xfs_info output (XFS)

### ext4

```bash
sudo debugfs -R 'dump <8> sda3.journal' /dev/sda3
sudo dumpe2fs /dev/sda3 > sda3.dumpe2fs
python ./fjta.py -i sda3.journal
```

### XFS

```bash
sudo xfs_logprint -C rl-root.journal /dev/mapper/rl-root
sudo xfs_info /dev/mapper/rl-root > rl-root.xfs_info
python ./fjta.py -i rl-root.journal
```

## Tested on

- Ubuntu 24.10 with kernel 6.8.0-88
- Rocky Linux 9.4 with kernel 5.14.0-570.49.1.el9_6.x86_64

## Supported Formats

- RAW
- EWF
- VMDK
- VHD / VHDX
- Directly filesystem (ext4 and XFS partitions)

## Contributing

Contributions are welcome! If you wish to contribute, please fork the repository and create a feature branch. Pull requests are greatly appreciated.

## Limitations

- FJTA is still under development, so some filesystem data may not be available for analysis.
- FJTA can analyze only ext4 and XFS version 5 (inode version 3).
- FJTA does not support LVM.
- Only ext4 journals stored with "data=ordered" are supported. data=ordered is the default journaling mode in most Linux distributions.
- Fast commit on ext4 is not supported.
- External journals are not supported.

## Author

[Minoru Kobayashi](https://x.com/unkn0wnbit)

## License

FJTA (Forensic Journal Timeline Analyzer) is released under the [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0). See the LICENSE file for more details.

fjta

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README