An Investigation Into Reverting Image Enhancements Using Deep Learning

As part of my individual project, I decided to tackle the issue of reverting image enhancements with the use of a Generative Adversarial Network (GAN) as my approach. The work undertaken and the results synthesised from the investigation can be found in my dissertation here.

Setup

Pre-requisites

• Tensorflow 1.4.1

Recommended

• Linux with Tensorflow GPU edition + cuDNN

Getting Started

# Clone this repository:
git clone git@github.com:fjmillman/image-reversion.git
 
# Change directory into the repository
cd image-reversion
 
# Download the datasets
bash download-datasets.sh
 
# Train the model
python main.py --input_dir datasets/{dataset}/train
    --output_dir datasets/{dataset} --mode train
 
# Test the model
python main.py --input_dir datasets/{dataset}/test
    --output_dir datasets/{dataset}/results --mode test
    --checkpoint datasets/{dataset}

Testing the outputs

The outputs of testing can be compared to see whether the model has successfully managed to revert the image enhancements. Two different operations can be run to find the image similarity metric averages of the dataset or to get a sum of absolute difference heatmap between each pair of images.

Replace operation with 'metrics' or 'heatmap'

python test.py --image_dir_a datasets/{dataset}/results/targets
    --image_dir_b datasets/{dataset}/results/outputs
    --output_dir datasets/{dataset}/results --operation {operation}

The Dataset

I generated a supervised dataset of image pairs using the MIT-Adobe FiveK dataset created by Bychkovsky et al.. It consisted of a set of 100 image pairs divided into the same 75 training image pairs and 25 testing image pairs for 6 different datasets. Each dataset represents a variation of image enhancements applied to an image with two colour enhancements, two vignette enhancements, and two combining the two enhancements together.

These 6 datasets were also combined into one superset for the purpose of training the model on all of the datasets at once to explore how the GAN could handle this.

I also generated an additional sub-dataset consisting of 6 different variations of colour enhancements and vignette enhancements applied to 10 different images which would be used to explore the GAN's ability to handle these variations.

The dataset can be downloaded here.

The Vignette Dataset

As part of my investigation, I discovered a limitation of the GAN in its ability to handle the removal of a deeper depth of vignette. I generated a dataset that could be used to explore this limitation consisting of the same 100 image pairs split into 75 training image pairs and 25 testing image pairs for 10 different datasets each with an incrementing depth of vignette.

The vignette dataset can be downloaded here.

The Results

Testing the Removal of Individual Enhancements

The first part of my investigation looked at training the model up on each of the six datasets individually.

Testing the Removal of Combined Enhancements

The second part of my investigation looked at training the model up on the combined superset.

Testing the Removal of Different Enhancements

The final part of my investigation looked at testing the test dataset on the model trained using the combined superset.

Acknowledgements

This implementation is based on a port of pix2pix from PyTorch to Tensorflow which was written by affinelayer.