KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #developer-tools #ai-agent #codex #typescript #cli #deepseek-harness #claude-code #ai-agents #mcp #rust

Text2Colors

★ 179 Open GitHub ↗

Code for "Coloring with Words: Guiding Image Colorization through Text-based Palette Generation" - ECCV 2018

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

--------------------------------------------------------------------------------
This repository provides a PyTorch implementation of [Text2Colors](https://arxiv.org/abs/1804.04128). Text2Colors is capable of producing plausible colors (or color palette) given variable length of text input, and colorize a grayscale image based on the generated color palettes.

&nbsp;

## Paper
[Coloring with Words: Guiding Image Colorization through Text-based Palette Generation](https://arxiv.org/abs/1804.04128) 
[Hyojin Bahng*1](https://github.com/hjbahng), [Seungjoo Yoo*1](https://github.com/sjooyoo), [Wonwoong Cho*1](https://github.com/wonwoongJo), [David Keetae Park1](https://github.com/heykeetae), [Ziming Wu2](https://github.com/jimmy-ng), [Xiaojuan Ma2](https://www.cse.ust.hk/~mxj/), and [Jaegul Choo1](https://sites.google.com/site/jaegulchoo/)
 
1Korea University &nbsp;&nbsp; 2Hong Kong University of Science and Technology
 
ECCV, 2018. (* equal contributions)

&nbsp;
&nbsp;

## Model Description
### Text-to-Palette Generation Networks (TPN) and Palette-Based Colorization Networks (PCN)
<img width="100%" src="PNG/model1.PNG" />
Overview of our Text2Colors architecture. During training, generator G0 learns to produce a color palette (y hat) given a set of conditional variables (c hat) processed from input text x = {x1, ···, xT}. Generator G1 learns to predict a colorized output of a grayscale image (L) given a palette (p) extracted from the ground truth image. At test time, the trained generators G0 and G1 are used to produce a color palette from given text, and then colorize a grayscale image reflecting the generated palette.

The model architecture of a generator G0 that produces the t-th color in the palette given an input text x = {x1, ···, xT}. Note that randomness is added to each hidden state vector h in the sequence before it is passed to the generator

&nbsp;
&nbsp;

## Palette-and-Text (PAT) dataset

We open our manually curated dataset named Palette-and-Text(PAT). PAT contains 10,183 text and five-color palette pairs, where the set of five colors in a palette is associated with its corresponding text description as shown in Figs. 2(b)-(d). The text description is made up of 4,312 unique words. The words vary with respect to their relationships with colors; some words are direct color words (e.g. pink, blue, etc.) while others evoke a particular set of colors (e.g. autumn or vibrant).
<img width="100%" src="PNG/pat.PNG" />
Statistics and samples of PAT dataset: (a) the number of data items with respect to their text lengths. On the right are examples that show diverse textpalette pairs in PAT. Those text descriptions matching with their palettes include (b) direct color names, (c) texts with a relatively low level of semantic relations to colors, (d) those with a high-level semantic context.

**For the use of PAT dataset for your research, please cite our [paper](https://arxiv.org/abs/1804.04128).**

&nbsp;

## Results
<img width="100%" src="PNG/test_palettes.png" />
<img width="100%" src="PNG/qualitative.png" />

## Prerequisites
* [Python 3.6+](https://www.continuum.io/downloads)
* [PyTorch 0.4.1](http://pytorch.org/)
* [Scikit-Image 0.13.1](http://scikit-image.org/)

&nbsp;

## Usage

#### 1. Clone the repository
```bash
$ git clone https://github.com/awesome-davian/Text2Colors.git
$ cd Text2Colors/
```

#### 2. Dataset & Libraries install
```bash
$ bash install_pre.sh
```

#### 3. Train 
##### (i) Training Text-to-Palette Generation Networks (TPN) with PAT data

```bash
$ python main.py --mode train_TPN
```

##### (ii) Training Palette-Based Colorization Networks (PCN) with CUB-200-2011* data

```bash
$ python main.py --mode train_PCN
```
*Wah, Catherine, et al. "The caltech-ucsd birds-200-2011 dataset." (2011).

#### 4. Test
##### (i) Testing TPN

```bash
$ python main.py --mode test_TPN
```

##### (ii) Testing Text2Colors

```bash
$ python main.py --mode test_text2colors
```

## Citation
If this work is useful for your research, please cite our [paper](https://arxiv.org/abs/1804.04128).

&nbsp;

Text2Colors

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README