Solving sudoku from images using Computer Vision and Deep Learning.
Working demo of project can be tested
-
Input the sudoku image.
-
Apply Thresholding to image.
-
Select the biggest contour with four corners.
-
From 4 Point get the Perspective transform.
-
Divide the wrapped image into 81 cells.
-
Extract each cell in the wrapped image and chop it borders as it may contain some white pixels.
fig: cell extracted from wrapped image.
fig: cell with chopped borders. -
Center the digit on the cell.
-
Predict the number on cell using Neural Network.
-
Use BackTracing Alogrithm to solve sudoku.
-
Display the result.
At first i tried on MNIST dataset for number prediction. But the model was having some issues differentiating between number 1 and 7 and sometimes 6 and 8. So i used TheChars74K dataset and apply some manual filtering and applied necessary processing dataset so that there is no issue differentiating number 1 and 7 and also 6 and 8.The dataset is splitted into 80-20 train-test split with following operations applied while traning :
test_path = data_dir+'\\test\\'
train_path = data_dir+'\\train\\'
image_shape = (28,28,1)
traindata_gen=ImageDataGenerator(
rotation_range=10,
rescale=1./255,
width_shift_range=0.1,
height_shift_range=0.1,
shear_range=0.1,
zoom_range=0.1,
fill_mode='nearest'
)
testdata_gen=ImageDataGenerator(
rescale=1./255)
traning_set=traindata_gen.flow_from_directory(train_path,
target_size =image_shape[:2],
batch_size = batch_size,
color_mode="grayscale",
class_mode = 'categorical')
testing_set=testdata_gen.flow_from_directory(test_path,
target_size = image_shape[:2],
batch_size = batch_size,
color_mode="grayscale",
class_mode = 'categorical',
shuffle=False)
1 . https://www.youtube.com/watch?v=uUtw6Syic6A&t=222s
2. https://github.com/manpreet1130/RealTime-Sudoku-Solver
3. Backtracing Algorithm : https://techwithtim.net/tutorials/python-programming/sudoku-solver-backtracking/
4. Text Centering : https://medium.com/@o.kroeger/tensorflow-mnist-and-your-own-handwritten-digits-4d1cd32bbab4